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1. Introduction 

While the use of measurement uncertainty in dimen- 
sional measurements, particularly uncertainty repre- 
sented according to the ISO Guide to the Expression 
of Uncertainty in Measurement (GUM), is a well- 
established practice among national metrology institutes 
(NMls), is a required basis for commercial laboratory 
accreditation, and is being specified in international 
product standards, such use is rare in U.S. industry 
and problematic where used, being unsupported by 
documentary standards that could provide specific 
practical guidance. To help address the problem. Special 



Committee H213 of the Board of Standards of the 
American Society of Mechanical Engineers (ASME) 
and the Manfacturing Engineering Laboratory of the 
National Institute of Standards and Technology (NIST) 
co-sponsored the workshop which this report describes. 
The conference organizers were: Dr. Gregory Hetland, 
Hutchinson Technology Incorporated; Dr. Jay Raja, 
University of North Carolina, Charlotte; and Dr. Dennis 
Swyt, NIST, who acted as conference moderator. 

2, Purpose of Workshop 

The purpose of the 2 1/2 day workshop was two-fold. 
The first purpose was to inform participants on trends 
and problems associated with the use of measurement 
uncertainty in industrial applications from a variety of 
perspectives, particularly problems addressable by 
documentary standards-based recommended practices. 
The second purpose was for participants to develop 
recommendations, particularly to the documentary 
standards community, on potential solutions to 
identified problems. 

3, Participants of the Worlishop 

The intended audience for the workshop was 
industrial and laboratory metrologists, engineers, and 
managers with interests in the use of statements of 
uncertainty for results of dimensional measurements for 
any purpose. Notification of potential attendees was 
via NIST's conference announcement process, the 
NCSL-l' newsletter, and a broad mailing to mem- 
bership lists of ASME standards committees, the 
American Society for Precision Engineering, and NIST 
dimensional calibration customers. Fifty-six persons 



See Appendix A for explanations of acronyms. 
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from the following organizations participated in the 
workshop: 

A2LA 

Adcole 

AHInc 

Axeon 

Boeing 

Bosch Brakes 

Caterpillar 

Daimler-Chrysler 

Dyncorp 

Fairchild 

Gagedoctor 

Hutchinson Technology 

Mahr Federal 

Mitutoyo 

NIST 

NRC-Canada 

Pratt & Whitney 

S Cal Edison 

UNC Charlotte 

US Air Force 

US Army 

Worcester Polytech 

Wyle Laboratories 

4. Structure of Workshop 

The workshop was structured to provide participants 
with reports on practices and problems in the use of 
uncertainty in dimensional measurements and an oppor- 
tunity to make recommendations on potential solutions 
to the problems identified. The first day was devoted to 
presentations on practices and problems in the use of 
uncertainty in dimensional measurements from differ- 
ent points of view. The following list shows the topic, 
speaker, and affiliation for each of the 12 presentations: 



U.S. national documentary T.Estler 



NIST/ASMEB89.7 



U.S. national documentary 

Measuring-instrument 
supplier 

Aircraft manufacture 

Auto manufacture 

Computer equipment 
manufacture 

Microelectronics 
manufacture 

Industry standards 
laboratories 

International documentary 
standards 



J. Salsbury Hutchinson Technology 

J. Salsbury Mitutoyo 

B. Parry Boeing 

J. Fallert Daimler-Chrysler 

J. Buttress Hutchinson Technology 

R. Scace NIST/OMP 

K. Jaeger NCSL-I 

T. Charlton ASME H213 



standards 






National Measurement 
Institute 


J. Decker 


NRC-Canada 


R&D on coordinate 


S. Phillips 


NIST 


metrology 






Laboratory accreditation 


S. Doty 


NVLAP 


VIM-GUM 


T.Estler 


NIST/JCGM 



The second day of the workshop was devoted succes- 
sively to breakout sessions for identification of problems 
in the use of uncertainty in dimensional measurements, 
a plenary session for review of problems identified, and 
breakout sessions for development of proposed solutions 
to problems. Breakout sessions were facilitated by 
T. Estler, J. Kramar, S. Phillips, J. Stoup, and D. Swyt 
of NIST. The third (half) day was devoted to plenary- 
session development of rank-ordered recommendations 
on problems identified, including follow-up actions for 
individual workshop participants. 

5. Results of Workshop 

The following are the seven resulting rank-ordered 
statements of identified problems, proposed solutions, 
and recommended actions to be coordinated by ad hoc 
workshop follow-up committee lead by workshop 
organizers. 

5.1 Steering Group to Lead in Measurement 
Uncertainty Education 

Problem: While (1) international standards bodies are 
developing specifications requiring use of measurement 
uncertainty, (2) domestic manufacturers anticipate 
growing need for their use and (3) calibration laborato- 
ries seeking accreditation must use them to establish 
required traceability, manufacturers point to the lack of 
a body of people who are trained to understand and 
knowing how to use uncertainty as these specifications 
require. Laboratories, on the whole, are afraid of the 
term "uncertainty budget" because they do not know 
how to do calculate one, and educators note a lack of 
easily accessible resource materials to educate properly 
current and future practitioners in measurement uncer- 
tainty. Further, while virtually everyone, from NMI to 
industrial practitioner, needs to produce statements of 
uncertainty, many do not understand the instruments 
and measurement processes well enough and do not 
have the resources to acquire the understanding needed 
to do so. In addition, no single entity has overall respon- 
sibility for the problems of "training and education" in 
the area of uncertainty in dimensional measurements. 
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Solution: Under the leadership of an ad-hoc steering 
group: 

• Establishment of a group (possibly composed of 
high-level managers from industry, government 
agencies, and universities) with the mission to 
promote the long-term sustainable educationa 

efforts in the use of uncertainty in measurements, 
particularly dimensional measurements. 

• Establishment of a group (possibly within ASME 
B89.7, NCSLI, NIST, or universities) with the 
mission to catalogue currently available educational 
resources for uncertainty issues. 

• Development of an "educational framework," that is, 
a comprehensive vision and scheme for education in 
measurement uncertainty, covering 

what needs to be known (general: statistics, cost/ 
benefit ratios, VIM/GUM, metrology, geometric 
dimensioning and tolerancing, mathematics, 
physics; industry-specific: heavy manufacturing, 
light manufacturing, precision manufacturing, 
semiconductor, electrical) 

who needs to know it (practitioners, assessors, 
financial managers, technical managers, NMIs, 
researchers, customers) 

who will fund it ( NSF, NIST, DOE, DOD, CCG, 
SME, ASQ, industrial consortia, students, industry 
lobby groups) 

how it can be provided (self-paced learning, 
commercial training classes, websites, free modules 
for university training) 

who will provide it (vendors, consortium of 
universities and national labs, professional societies, 
companies) 

when it should be provided (high school instruc- 
tions through bridge programs/internships; post- 
secondary education; vocational/trade schools, 
junior colleges, universities, just-in-time training, 
continuing education) 

how it will be validated/certified (multi-level 
certification by professional societies, e.g., ASME, 
SME, NCSLI, SAE, AMTMA, ASQ, vendors, 
colleges, universities). 

5.2 Consortium Devoted to Meeting Industrial 
Needs in Measurement Uncertainty 

Problem: The cost-burden on individual, especially 
small, companies is too great (at this time) for them to 
support the development of education, training, commu- 
nication, and standards-development infrastructure 
needed to effectively and promptly address the wide 
range of such problems in the use of uncertainty in 
measurements, particularly dimensional measurements, 
facing U.S. industry today. 



Proposed Solution: Establishment of an industrial 
consortium dealing with problems in the use of uncer 
tainty in measurements, particularly dimensional mea- 
surements, in order to leverage dollar investments by 
individual members of the consortium and to pursue 
other sources of funding, including from Congress, the 
NIST ATP Program, NSF and the like. 

5.3 Website for Measurement Uncertainty 
Documents 

Problem: While there are many documents on uncer- 
tainty in dimensional measurements, (including interna- 
tional and national standards, accreditation practices, 
and technical reports) of potential value to a broad 
range of possible users, these documents are scattered 
and not easily accessible to many, with no easy and 
efficient means by which one may find out about them. 

Proposed Solution: Creation and operation by an ap- 
propriate institution of an "Uncertainty in Dimensional 
Measurements" web site focussed on the principal doc- 
uments on the subject. Proposed features of the website 
include: optional registration; a hierarchy of docu- 
ments; diagram of standards, and guides showing inter- 
relationships; synopses of documents; user reviews of 
documents; news of upcoming round-robins; user fo- 
rum; case studies; testimonials to cost savings and con- 
tinuous improvement; a roadmap on developments; and 
a primer for neophytes. 

5.4 Business-Case Justifications on Use of 
Measurement Uncertainty 

Problem: The business community requires cost jus- 
tification for allocating dollars to implement measure- 
ment uncertainty. For example, executives and man- 
agers need a basis for spending on training and devoting 
human resources to support analysis and use of mea- 
surement uncertainty and purchasing agents need a ba- 
sis to weigh the uncertainty versus cost in acquiring 
measuring instruments. 

Solution: Development by ASME B89.7 of technical 
reports that provide business-case justification for the 
use of uncertainty in measurements, including 

• "problem statement paper covering ROI opportuni- 
ties" dealing with 

decision rules (false rejects, false accepts) 

calibration frequency extended life of measure- 
ment systems 

product warranty risks quantified 

improved process to achieve competitive 
advantage 

allocation of limited resources for highest ROI 
(CMM, lab, fixture, sampling, cleaning) 

recovery of yields loss 
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• ROI case studies as published technical reports 

ROI Case Study 1 on selecting measuring equip- 
ment (including system, room, cleaning, allocating 
resources; impact of required infrastructure from 
industrial labs to NIST; determining calibration 
frequency and control parameters; reducing buyer/ 
seller disagreement) 

ROI Case Study 2 on dealing with measurement 
uncertainty with respect to tolerances at the design 
stage (including recovering yield loss through 
decision rules, process control and impacts of 
measurement, e.g.. Motorola Six-Sigma Quality) 

ROI Case Study 3 on Failure Mode and Effects 
Analysis (Catastrophic failures, e.g.. Shuttle 
Challenger, Ford-Firestone, Dodge Minivan latches, 
seatbelt recalls and lawsuits, scrap of high-cost 
products). 

5.5 Accreditation-Related Standard Methods 
on Uncertainty 

Problem: In the area of accreditation (which is 
defined broadly to include accreditation, certification, 
and auditing), there is an absence of documented 
standard methods related to measurement uncertainty 
for use by both auditors of laboratories and laboratories 
being audited. While many institutions — as part of a 
system — create, invoke, or require procedures on the 
use of measurement uncertainty and end-users of such 
procedures may have to deal with inconsistent require- 
ments, no single institution has responsibility in the 
system for producing a single, self-consistent, techni- 
cally-sound set of standard procedures that may be used 
by all. 

Proposed Solution: The proposed solution is the 
coordinated development by ASME, NCSLI, NIST, 
NACLA (and other institutions with shared responsibil- 
ity in the area of measurement uncertainty) of docu- 
mented standard methods for: itemizing and listing 
contributions to uncertainty that must be addressed; ap- 
propriately defining "scope of accreditation," e.g.. for 
coordinate metrology; evaluating the credibility of 
uncertainty statements; proficiency testing of on-site 
(non-laboratory) calibrations; formatting uncertainty 
budgets; and demonstrating traceability in practice. 

5.6 Addressing of Specific Technical Issues 
by Standards Committee 

Problem: There is an array of problems associated 
with current and needed documentary standards govern- 
ing the use of uncertainty in dimensional measure- 
ments, including: 

The GUM, its complexity for most people to use, 
and its unsuitability for economic decision-making 



The establishment of credibility and validity of 
uncertainty statements 

Identification of significant input quantities 

Inconsistency in reporting format 

Inconsistency in terminology/vocabulary and 
interpretation 

Decision rules connected to economics 

Calibrated instruments used for unlike measur 
ands and/or extended conditions 

Evaluation of measurement uncertainty for 
calibrations based on a single observation 

Measurands that are insufficiently defined by 
some standards (e.g., ISO 1101) 

Lack of consensus standards on some instrument 
performance specifications/tests . 

Proposed Solution: 

• Development by the international committee respon- 
sible for the GUM (JCGMAVG1/SC4) of models for 
measurement uncertainty that are simple, general, 
and usable by industry 

• Development by the U.S. national standards commit- 
tee responsible for uncertainty in dimensional mea- 
surements (ASME B89.7) of documentary stan- 
dards, including formal standards or technical 
reports, that 

provide a simplification of the GUM more usable 
by industry than the GUM itself (B89.7.3.2) 

define a basis for establishing the credibility and 
validation of statements of uncertainty in dimen- 
sional measurements (B89.7.3.3) 

identify the significant input quantities to uncer- 
tainty in dimensional measurements (B89.7.3.2) 

standardize the reporting format for statements of 
uncertainty, uncertainty budgets, expected sources 
of uncertainty, and the criteria for consideration of 
those expected sources within the reporting format 
(B89.7.3.1) 

provide decision rules connected to economics for 
the use of measurement uncertainty in, for example, 
determining conformity of manufactured parts or 
measuring instruments to specifications (B89.7.3.1) 

deal with instruments that are calibrated for one 
measurand under one set of measurement conditions 
being used for other measurands and/or other condi- 
tions [see "Careful Consideration of the Calibration 
Concept," M. S. Levenson et al., J. Res. Natl. Inst. 
Stand. Technol. 106, 371 (2001)] deal with evaluat- 
ing measurement uncertainty for calibrations or 
measurements based on a single observation 
(measurement), rather than repeated ones, by GUM 
Type B assessment (B89.7.4) 
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• Action by institutions with shared responsibihty in 
use of measurement uncertainty (ASME, ASQ, 
NCSLI, NIST, NRC-Canada, . . . ) to 

promote use of International Vocabulary of 
Metrology (VIM) 

identify inconsistencies in terminology, vocabu- 
lary, and interpretation within their respective 
domains 

identify dimensional-metrology instruments lack- 
ing consensus 

standards on uncertainty-related performance 
specifications and testing 

inform appropriate standards committees of such 
inconsistencies in terminology and absence of 
needed instrument performance tests. 

5.7 Establishment of Sound Practice on Teclinical 
Qualification of Assessors 

Problem: Variability in technical qualifications of 
laboratory accreditation assessors and/or technical 
experts resulting in inconsistencies among accredited 
laboratories compromises (in some cases) the actual or 
perceived integrity of the laboratory accreditation 
process. 

Solution: Development and application by NACLA 
(National Cooperation for Laboratory Accreditation) of 
sound practice and a uniform method for the specifica- 
tion and verification of the technical qualifications of 
assessors, whether they be the assessors themselves or 
the technical experts that operate with formal assessors 
in the laboratory accreditation process. 

5.8 Operational Follow-Up to Recommendations 

As an operational follow-up recommendation, the 
participants in the workshop together recommend the 
establishment by the organizers of the workshop of an 
"Ad-Hoc Group on Measurement Uncertainty in 
Dimensional Measurements" made up of the organizers, 
volunteers from among the other participants of the 
workshop, and other interested parties to pursue by 
appropriate actions the seven other recommendations of 
the workshop. 

The participants of the workshop also suggest: 

a prompt follow-up meeting of workshop partici- 
pants and others 

posting of workshop presentations immediately 
on the NIST/PED website 

a publicity article on the workshop by the 
organizers in appropriate journals such as a 
Conference Report in the Journal of Research 
of NIST. 



The following workshop participants volunteered or 
were nominated to take follow-up actions on the various 
recommendations of the workshop: 

(1) establishment of steering group on education in 
measurement uncertainty, including proposal to 
NCSL to take special role in steering group Steve 
Stahley, Walt Lehmus, Jim Ferguson, Ed Morse 

(2) establishment of a consortium on meeting industrial 
needs Rob McNaughton (to work, not lead) 

(3) establishment of a website on documentary infor- 
mation Jay Raja (nominated) 

(4) development of business-case justifications on use 
of uncertainty John Buttress representing ASME 
B89.7 committee 

(5) development of accreditation-related standard 
methods (workshop organizers by default) 

(6) addressing specific technical issues by a standards 
committee (workshop organizers by default) 

(7) establishment of sound practice on qualification of 
assessors (workshop organizers by default). 

6, Appendix A. Acronyms 

AMTMA American Measuring Tool Manufacturers 
Association 

ASME American Society of Mechanical Engineers 

ASQ American Society for Quality 

BIPM Bureau Internationale des Folds et Mesures 

CCG Calibration Coordinating Group 

CMM Coordinate Measuring Machine 

DOD Department of Defense 

DOE Department of Energy 

FMEA Failure Mode and Effects Analysis 

GUM ISO Guide to the Expression of Uncertainty 

in Measurement 

HTI Hutchinson Technology Incorporated 

ISO International Organization for Standardiza- 

tion 

JCGM BIPM Joint Committee for Guides in 
Metrology 

NACLA National Cooperation for Laboratory Ac- 
creditation 

NVLAP National Voluntary Laboratory Accredita- 
tion Program 
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NCSL-I National Conference of Standards Labora- 
tories International 

NIST National Institute of Standards and Technol- 
ogy 

NMI National Metrology Institute 

NRC National Research Council of Canada 

NSF National Science Foundation 

OMP NIST Office of Microelectronic Programs 

PED NIST Precision Engineering Division 

ROI Return on Investment 

SAE Society of Automotive Engineers 

SME Society of Manufacturing Engineers 

UNC-C University of North Carolina — Charlotte 

VIM International Vocabulary of Basic and 

General Terms in Metrology. 
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